Larry Wasserman Low Assumptions , High Dimensions

These days, statisticians often deal with complex, high dimensional datasets. Researchers in statistics and machine learning have responded by creating many new methods for analyzing high dimensional data. However, many of these new methods depend on strong assumptions. The challenge of bringing low assumption inference to high dimensional settings requires new ways to think about the foundations of statistics. Traditional foundational concerns, such as the Bayesian versus frequentist debate, have become less important. 1. In the Olden Days There is a joke about media bias from the comedian Al Franken: “To make the argument that the media has a leftor right-wing, or a liberal or a conservative bias, is like asking if the problem with Al-Qaeda is: do they use too much oil in their hummus?” I think a similar comment could be applied to the usual debates in the foundations of statistical inference. The important foundation questions are not ‘Bayes versus Frequentist’ or ‘Objective Bayesian versus Subjective Bayesian’. To me, the most pressing foundational question is: how do we reconcile the two most powerful needs in modern statistics: the need to make methods assumption free and the need to make methods work in high dimensions. Methods that hinge on weak assumptions are always valuable. But this is especially so in high dimensional problems. The Bayes-Frequentist debate is not irrelevant but it is not as central as it once was. I’ll discuss Bayesian inference in section 4. Our search for low assumption, high dimension methods is complicated by the fact that our intuition in high dimensions is often misguided. In the olden days, statistical models had low dimension d and large sample size n. These models guided our intuition but this intuition is inadequate for modern data where d > n. An analogy from physics is helpful. Physics was initially guided by simple thought (and real) experiments about falling apples, balls rolling down inclined planes and moving objects bumping into each other. This approach guided physics successfully for a while. But modern physics (quantum mechanics, fields